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ABSTRACT 

In the last decade, several attempts have been made 
to relate item response theory (IRT) models to latent class analysis 
(LCA) models. One of these attempts is the solution-error 
response-error (SERS) model, an LCA model in which the structure of 
the latent class probabilities is explained by a one-dimensional 
loglinear Rasch model. In this paper, the SERE model is generalized 
to models for polytomously scored latent states that may be explained 
by a multidimensional latent space. In this generalized SERE model a 
distinctiojj is made between some well-defined latent states in which 
the subject has a certain amount of knowledge of the answer. The 
probability that the subject is in a certain state is assumed to be 
governed by the multidimensional polytomous latent trait model. The 
relationship between the latent states and the observed answers is 
described by conditional probabilities. Two appendixes present items 
from a clinical pathology test and a proof of the collapsed 
generalized SERE model. Five tables and four figures illustrate the 
discussion, (Contains 38 references,) (SLD) 
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Abstract 



In the last decade, several attempts have been made to relate item response theory (IRT) models 
to latent class analysis (LCA) models. One of these attempts is the solution-error response-error 
(SERE) model; a LCA model in which the structure of the latent class probabilities is explained 
by a one-diroensional loglinear Rasch model. 

In this paper the SERE model will be generalized to models for polytomously scored 
latent slates that may be explained by a multidimensional latent space. 
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Generalizations of the 
Solution-Error Response-Error Model 

Introduction 

For measuring individual differences, a distinction can be made between measurements on a 
discrete qualitative latent trait and measurements on a continuous quantitative scale. The latent 
class analysis (LC A) model, where it is assumed that subjects belong to different latent classes, 
belongs to the first case (Bartholomew, 1987; Lazarsfeld & Henry, 1968; Mooijaan, 1978). 
Whereas, the item response models (IRT) belongs to the second case. Some well known 
examples of IRT models are the Rasch (1960/1980) model and the two and three-parameter- 
logistic or normal ogive models (Lord, 1980; Lord & Novick, 1968). In the last decade, several 
attempts have been made to relate IRT models to LCA models (Bock & Aitkin, 1981; Dayton & 
Macready. 1980; Formann, 1985; Kelderraan, 1988a, 1989; Kelderman & Macready, 1990; 
Mislevy & Verhelst, 1990; Yanianioto, 1987, 1988). In this paper one of these attempts will be 
discussed: the solution-error response-error (SERE) model of Kelderman (1988a). 

In the SERE model a distinction is made between a "Know" state in which the subject 
has complete knowledge of the answer and a "Don't know" state. The probability that the subject 
is in the "Know" state is assumed to be govemeJ by the Rasch model. Further, it is assumed that 
if the subject is in the "Don't know" state, the subject will choose an alternative, where the 
attractiveness may be different for different alternatives, including the correct one. The SERE 
model can be formulated as an (incomplete) LCA model, where each latent class corresponds to 
an idealized response pattern. The relations betw^n these idealized responses are explained by 
the loglinear version of the Rasc^ model (Cressie & Holland, 1983; Duncan, 1984; Kelderraan, 
1984; Tjur, 1982). 
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All SERE models considered in Kelderman (1988a) deal with a one-dimensional 
continuous latent trait. In many testing situations, however, we may have to deal with a two - or 
more - dimensional latent space. For example, consider a version of the American Society of 
Clinical Pathologist (ASCP) Microbiology-TesL In Appendix A.l some items of this test are 
presented. Content experts have hypothesized that although each item of this ASCP test has one 
correct alternative, incorrect responses might often be chosen after cognitive activities similar to 
those necessary to arrive at the correct response. They presumed further that "Applying 
Knowledge", "Selecting Action", "Calculating", "Correlating Data" and "Evaluating Problem" 
are the involved cognitive processes in answering tlie items. For instance, it was assumed that for 
item 1 1 of Appendix A. 1 the correct answer (d) involved two applications of knowledge, 
whereas answer c only one. For giving the correct answer c on item 20 it was assumed that the 
subject had to use the cognitive process "Evaluating Problem" two times and the cognitive 
processes "Applying Knowledge" and "Selecting Action" ones. 

So, in general, producing one answer may require quite another ability from the 
examinee than to produce another answer. Or some responses may require the repeated 
application of an ability, whereas others may require only a single application of the same 
ability. In this paper the SERE model will be generalized to models for polytoraously scored 
latent states that may be explained by a multidimensional latent space. Maximum likelihood 
estimates of the parameters of this generalized SERE (GSERE) model can be obtained by 
solving the likelihood equations by the iterative proportional fitting (IPF) algorithm of Goodman 
(1974b). 

In what follows, the GSERE model will be formulated. The estimation method and 
goodness-of-fit tests are described, and the question of identifiabilily is discussed. 
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The Generalized Solution-Error 
Response-Error Model 

Suppose that each subject, randomly drawn from a population of subjects, responds to k test 
items, wheie the answer to item j may be any of the rj responses, denoted by yj (yj=l,...,rj). Let 
Xj (xj=0,...,Sj) indicate the latent slate of the subject. For example, all the items of the ASCP 
Microbiology Test have four possible responses (i.e., rj=4) and may have three latent states: 
"Don*t know", "Partial knowledge" and "Complete knowledge". It is assumed that if the subject 
is in the "Don't know" state (xj=0), (s)he will choose one of the alternatives. If the subject is in 
the "Partial knowledge" state (xj=l), (s)he will choose one of tlie alternatives that might be right 
given the subject's partial knowledge of the answer. If the subject is in the "Complete 
knowledge" state (Xj=2), (s)he will choose the right alternative. The random variables with 
values yj and Xj are denoted by Yj and Xj (j=l,...,k). The relationship between the latent stale Xj 
and the observed response yj is described by the conditional probability 

This conditional probability will be referred to as the attraction parameter of item j. 

In the generalized solution-error response-error (GSERE) model it is assumed that the 
latent states are not governed by the Rasch (1960/1980) model, but by the more general 
multidimensional polytomous latent trait model (MPLT) by Kelderman (1988b). In the MPLT 
model it is assumed that the subject must perform certain cognitive operations to produce a score 
Xj on item j. See for instance the example in the previous section. Each operation depends on a 
certain proficiency on a latent trait. Let Bjq(x) be a non-negative weight associated with the 
dependence of response x on item j on the latent trait q. Furthermore, let 8jq(x) be the difficulty 
parameter of the response x on item j related to latent trait q (q=l v), 9q be a value of the 



3 



Generalized SERE Model 
5 

subject on the latent ability continuum and 9 = (8 i,..M6y) be the vector of ability values. The 
probability that the subject has a response xj on item j can be written as (Kelderman. 1988b) 

expfZ (eq.5jq(Xj))Bjq(Xj)} 

(2) P(x;ie) = 

^ lexpd (eq.5jq(z))Bjq(z)} 

z q 

Assuming local independence of Xj and Yj given the latent trait vector 0 and Xj, respectively, 
the probability of choosing response yj is equal to 

(3) P(yjie)= I p(yjixj.e)P(xjie) 

= J {*^-*yj}exp{X (eq.8jq(xj))Bj^^^^^ 

where 5j = (5j j»..M5jv), 

C(e.6j) = I exp { I (Bq - 5jq(z))Bjq(z)} 

As Kelderman (1988b) has shown, by specifying the scoring weights Bjq(.), different models can 
be chosen for the dependence of the latent states on the latent traits. To illustrate the main idea of 
this paper, in the following one specific MPLT model will be considered; the multidimensional 
partial credit (MPCM) model However, it may be clear that the contents of this paper is also 
valid for other kinds of MPLT models. 

The scoring weights for a MPCM model, where each step depends on a different latent 
trait, are depicted in Figure 1(a). 
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Insert Figure 1 here 



The "Complete knowledge" state (x=2) has scoring weight Bj j(2) = I on the first trait and 
scoring weight Bj2(2) = 1 on the second. The "Partial knowledge" state (x=l) has scoring weight 
Bj I ( 1 ) = 1 on the first trail, whereas the "Don*i know" state (x=:0) has scoring weights zero. 
Using (2) and the scoring weights Bjq(.) of Figure 1(a), the probability that the subject has a 
latent state xj on item j can be written as 

exp( r*(eq-8jq(xj))} 

(4) P(xjie) = — 

z q=l ^ ■'^ 

Adding a constant c to 5jq(l) and subtracting it from 8jq(r) (l^i^tr^j) does not change the 
model in (4). By setting the difficulty parameters of the same response equal to each other (i.e., 
5jq(xj) = 5jq for xj = 0,1,2 and all q), this in determinancy can be removed (Kelderroan, 1988b). 
From (4) and the assumption of local independence of the Xj's given the latent trail vector 6, it 
follows that the simultaneous distribution of X given 6 is 

(5) P(xl9) = exp{ I (Bq tq . 1 Bjq(xj) 5jq)} H C(e,5j)-^ 
where 

C(0,5j)=Iexp{ I (Bq-Sjq)} 
z q=l 
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and 



lq = IBjq(xj) q=l V 



As all other MPLT models, ihe MPCM model is a exponential family model and t=(ti,...»lv) is ^ 
sufficient statistic for the latent trait vector G (Kelderman, 1988b). 

Let I mean the summation over all possible latent stale patterns x^xj ,...,xj,). Using (1), 

X 

(5) and the assumption of local independence of the observed responses yj, tiie marginal 
probability of y given 6 can be written as 



(6) P(ylO) = 2:P(yix,e)P(x!e) 

X 



= X { H ^'^J^j } exp{ I (Gq tfl - 5 Bjq(xj) 5jq)} 0 0(6,6^^ 
X J xjyj q H s J JH J J J 



Rearranging tiie terms of (4) and letting F(0) be the distribution function of tiie latent ability 
vector e, tiie marginal probability of tiie observed responses y can be written as an incomplete 
LCA model in the sense of Haberraan (1979) 



(7) P(y) = I * * ...<t>^ * ^ ^ 

X t XI Xk xjyi x^yk 
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with 

it>J = JexpCI 6q 1^) FJ C(G,8j)-ldF(6), 

X; X 
X q=l •'^ 

In this model, each value of the latent state vector x represents a latent class. Maxiraura 
likelihood estimates of the parameters of the GSERE model can be obtained by solving the 
maximum likelihood equations by the iterative proportional fitting (IPF) algorithm 
(Bartholomew, 1987; Goodman, 1974b; Haberman, 1979; Hagenaars, 1990). The overall 
goodness-of-fit of a model can be tested by tlie Pearson statistic or the likelihood-ratio statistic 
(see Haberman, 1979). Together with the question of identifiability, these two issues will be 
discussed in the next sections. But first, some applications of the GSERE model will be 
discussed. 



Applications of the GSERE Model 

In the previous section a GSERE model was formulated, where tlie parameters of the model were 
unrestricted, except for the usual restrictions pertaining to probabilities and conditional 
probabilities. In the present section, it is discussed how to modify the GSERE model in order to 
make it suitable for special applications. 

Generally, for each specific GSERE model we may defme the weights Bjq(.) and certain 
constraints on the attraction parameters for each item j. The choices of the weights may depend 
on the required latent trait abilities for the correct response. For example, the item "20-5-6=?" 
requires two subtraction operations for the correct response, so that we can choose the one- 
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dimensional partial credit model as depicted in Figure 1(b). But, for the item "V( 169-25)=?", 
where the two abilities are subtraction and taking the square root, we can choose the two- 
dimensional partial credit model as depicted in Figure 1(a). In Kelderraan (1988b) other possible 
choices of the scoring weights for the dependence of the latent slates on latent traits are 
discussed. 



By specifying the attraction parameters (1) as free, equal to each other or fixed to a certain value, 
a particular GSERE model is specified (Kelderman, 1988a; Westers & Kelderraan, 1992). In 
Figure 2 some examples of the attraction parameters for the GSERE model are depicted. Figure 
2(a) describes the situation of a perfect response process; the subject answers the item correctly 
(y=l) if (s)he can solve the problem (x=l) and gives a wrong answer (yj=2) if (s)he cannot solve 
the problem (x=0). One case, where the items are not necessarily answered incorrectly if the 
subject cannot solve the problem, is with multiple choice items. If the subject doesn't know the 
answer, (s)he will guess the most attractive alternative. The attraction parameters for this 
situation are depicted in Figure 2(b). In Figure 2(c) the situation is depicted for the case where 
more then one alternative is correct Generally, it is assumed that if the subject can solve the 
problem formulated by the item, (s)he will give the correct answer. But, a subject may fail to 
produce a correct answer, even if the subject was able to solve the problem. On the other hand, if 
(s)he is not able to solve the problem, it is impossible to produce the correct answer. Such a 
situation is depicted in Figure 2(d)» where P is the so-called omission error. In the case of the 
MPCM modeU we may assume that the attraction parameters are specified as depicted in Figure 



Insert Figure 2 here 



2(e). 
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Identifiability 

Whether the maximum likelihood estimates of the parameters of the GSERE model are unique 
depends on the identifiability of the model. A necessary condition for identifiability is, of course, 
that the number of independent parameters to be estimated does not exceed the number of cell 
frequencies minus one (i.e., (U rj)-l). Furthermore, if the MPLT model is not (locally) 
identifiable, then the GSERE model is also not (locally) identifiable. 

Generally, in MPL"" Tiodels identifying restrictions must be put on the model parameters. 
Therefore, in the paper of Kelderman ( 1988b) conditions are formulated, which ensures that the 
difficulty parameters of the MPLT model are not linearly dependent upon each other and upon 
the proportionality constants, respectively. 

Since the (G)SERE model can be formulated as an (incomplete) LCA model, Goodmans 
(1974a) sufficient condition for identifiability can be used for the identifiability of the GSERE 
model. Let M be the matrix consisting of the derivatives of the function (7) with respect to the 
parameters of the GSERE model. The number of rows of the matrix M is equal to (FI rj)-l and 
the numbe. of columns is equal to the number of parameters of the GSERE model. By direct 
extensions of a standard result about Jacobians, the GSERE model will be locally identifiable if 
the rank of the matrix M is equal to the number of columns. The rank of the matrix M can be 
evaluated by numerical methods. 

When the parameters of the GSERE model are not identifiable, various kinds of 
restrictions can be imposed upon the attraction parameters and/or the item parameters in order to 
make the GSERE model identifiable. For instance, by specifying the attraction parameters to be 
equal to each other or to be fixed at a certain value. 

Unidentifiability can be discovered by estimating the parameters a second time, this time 
using different initial estimates. In the case of unidentifiability both runs will give different 
parameter estimates. 
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Estimation Method 

Let m^yt expected number of subjects with latent state x, observed response y and 
sumscore t. As Kelderman (1988a), and Westers and Kelderman (1992) have showed, the 
parameters of the SERE model can be estimated by applying the iterative proportional fitting 
(IPF) algorithm. For the GSERE the IPF algorithm can also be used. One of the differences with 
the SERE model is that, depending on the number of the latent state categories, the number of 
latent classes may be quite large. Since the number of attraction parameters depends on the 
number of latent state categories and the number of item response categories, thi<i number may 
also be quite large. 

The maximum likelihood estimates of the parameters of the GSERE model can be 
obtained by solving the maximum likelihood equations by a two-step algorithm. In the first step 
of each iteration (i.e., the outer iteration), the attractiveness of the alternatives and the expected 
frequency distribution of the latent classes will be estimated. For the GSERE model the first step 
is similar to the first step of the estimation method for the parameters in the LX^A model 
(Goodman, 1974b). In the second step of each iteration (i.e., the inner iteration), the estimated 
expected distribution of the latent classes is fitted to the postulated loglinear model. From this 
distribution the difficulty parameters can be estimated. 

5.1 Outer Iteration 

As indicated before the GSERE model can be formulated as a LCA model, where each latent 
class represents a latent state vector x. Let 

(8) P(y) := 1 P(x) P(y ibc) ... P(y^\x) 

X 
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where P(x) is the probability of getting latent suie vector x and P(yjlx) is the conditional 
probability of choosing response yj given the latent stale vector x. The model in (8) is a LCA 
model in the sense of Goodman (1974b)» which means that the IPF algorithm for the general 
latent structure model, where the parameters in the model are unrestricted, can be used for 
estimating the expected frequency distribution of the latent classes and the attractiveness of the 
alternatives. 

As mentioned before, in the GSERE model it is assumed that the observed response yj 
depends only on the latent sute xj, therefore the conditional probabilities P(yjlx) are restricted in 
the following manner 

PCyjlx) = P(yjlxO = ojjyj 
J J Xj yj 

for all latent sute vectors x and x' with components x* - xj. The C?^^ can be obtained by 

A 

taking a weighted average of the estimates P(yjlx) obtained from the IPF algorithm, using 

A 

weights proportional to P(x) (Goodman. 1974b). So 

cD^j^j = { I^p(x) p(yjix)} / { I p(x)} , 
where L means the summation over all latent state vectors X=x with X;=x:. 
5.2 Inner Iteration 

As assumed in the second section of this paper, the latent probabilities P(x) are restricted in such 
a way that they satisfy an MPCM model. Knowing that the MPCM model is an exponential 
family model and that the sum score t is a sufficient statistic for the latent ability parameter 6 
(Kelderman, 1988b), the conditional distribution of X given t is by 
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P(xlt) = exp{I(pj(Xj)}/ga(p), 
j 

with 

g(t(p)= I exp{ I(p|(xj)}, 
xlt ^ ^ 

where 1 means summaiion over those values of the latent state vector x for which 
xlt 

(Z Bji(x),...X Bjy(x)) is equal to t, and the vector (p = (cp}(xi),...,(|)j^(xj.)) denotes the weighted 
sums over latent trails of the difficulty parameters 

V 

cpj(Xj) = -I 5jq(Xj) Bjq(Xj). 
q=l 

Letting m^^i be the estimated expected number of subjects with latent state x and sum score t we 
have 

(9) log m^t = at + I q>j(xj) 

where = - log( g(t»cp)/Nt) is a fixed normalizing constant, is the number of subjects with 
sum score t For the MPCM model the parameters (pj(xj) are specified by (pj(l) = Sjj and 
(Pj(2) = -8j ] - 8j2» respectively. 

The model in (9) is a quasi-loglinear model for an incomplete item response 1 x-.<x item 
response k x score 1 x...x score v contingency table. The table is incomplete since for certain 
given values of X, only one t is possible. Maximum likelihood estimates can be obtained by 
solving the maximum likelihood equations of the MPCM model These equations can be solved 
by IFF (Kelderraan, 1988b). The latent probabilities P(x) arc then adjusted to these maximum 
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likelihood esiimaies. In this way new latent probabilities P(x) are obtained that satisfied the 
postulated MPLT model and are used again in the next outer iteration. 

Goodness-of-Fit Test 

The overall goodness of fit of the GSERE model can be tested by the Pearson statistic or the 
likeiihood-ratio statistic. Both statistics are asymptotically distributed as chi square with degrees 
of freedom equal to the difference between the number of cells in the observed contingency table 
and the number of estimable parameters. However, if the expected counts become too small, the 
approximation of the distribution of the goodness-of-fit statistics by a chi-square distribution 
becomes bad (Koehler, 1977; Lancaster, 1961; Lamz, 1978). 

Using the difference in the likelihood-ratio lest statistics for two models (Bishop, 
Fienberg, & Holland, 1975; Rao, 1973), it can be tested whether an alternative model yields a 
significant improvement in fit over a special case of this alternative model. 

If two GSERE models are not proper subsets of each other, then Akaike's (1977) 
information criterion (AIC) or Raftery's (1986a, 1986b) Bayesian information criterion (BIC) 
can be used. AIC is defined as 

AIC=:G^-(ln 2)d 

where G^ is the likelihood-ratio lest statistic and d is the number of independent parameters in 
the GSERE model. The BIC index has In N (i.e., the sample size) instead of In 2, but is otherwise 
identical. For both indices, the first terra is a measure of badness of fit, whereas the second terra 
is a correction for overfilling due to the increasing bias in as the number of parameters in the 
SERE model increases. The GSERE with the minimum AIC or BIC value will be chosen to be 
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the besl fitting model. Computer programs by Hagenaars and Luijkx G-CAG, 1990) and 
Kelderman and Steen (L0G1M0» 1988) can be used to fit the model. 



Example 



For numerical reasons, the GSERE model is difficult to be routinely applied in large testing 
programs yeL Not the number of parameters of the model causes the problems, but the number 
of latent classes and the tables of observed and expected counts become too large for computer 
storage. One solution to this problem may be the use of the Division-by-Item (DBI) principle of 
Westers (1992). In this paper a maximum likelihood estimation method for the one-dimensional 
SERE model for a large set of items was described. This method was based on dividing the 
whole item set in several subsets. The computational problem boils down to the simultaneous 
estimation of the parameters of a set of smaller SERE models. This could be done because one of 
the properties of the SERE model was the collapsibility of the model. Since this property is also 
valid for the GSERE model (Appendix A.2), we can also use the DBI-principle for the 
estimation of the parameters of the GSERE model. However, before the GSERE model can be 
widely applied, further research is required to reduce the amount of computer storage. But if the 
generalization to the multidimensional latent spare is ignored, then, for a small number of items, 
the parameters of the GSERE model can still be estimated by the combined of the programs 
LCAC and LOGIMO, In this section this will be demonstrated with an example. 

The authors were allowed to analyse four one-dimensional four-choice items from a 
protected data base of the ASCP Medical Laboratory Test ASCP produces tests for certification 
of paramedical personnel The items in the Medical Laboratory Test measures the ability to 
perform medical laboratory tests. The test score is obtained by adding the number of correct 
items. 
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Content experts from ASCP have hypothesized that the cognitive process "Applying 
Knowledge" was involved in answering these four items. According to the assumptions of the 



Insert Table 1 here 



content experts, we postulated that there are three latent states: "Don't know", "Partial 
knowledge" and "Complete knowledge", with scoring weights equal to zero, one and two, 
respectively. This means that it is assumed that the latent states are governed by the one- 
dimensional partial credit model (OPCM) as depicted in Figure 1(b). Table 1 shows the 
hypothesized weights that content experts gave for each of the item responses on the cognitive 



Insert Figure 3 here 



process "Applying Knowledge". Those hypothesized scoring weights are translated into 
specificatior.s o^ the attraction parameters, which are depicted in Figure 3(a) through (d), where 
x=0 indicates the "Don't know" state, x=l the "Partial knowledge" state and x=3 the "Complete 
knowledge" sute, respectively. 

In this example the hypothesized model (H) will be compared which three alternative models. 
The first model (A j) is a GSERE model, where for each item all the attraction parameters arc unequal to 
zero (Figure 4). Model A2 is the same as Model A 1 , but where it is assumed that the correct alternative 
will be chosen if the subject is in the "Complete knowledge" state. Model A3 has not only the same 
assumptions as Model A2, but it also assumes that 04=Pi (i=U.M4). The specifications of the attraction 
parameters of Models A2 and A3 are also depicted in Figure 4. 
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Insert Figure 4 here 



In Table 2 the Likelihood-ratio test statistics, the Akaike's information criteria and Raftery's 
Bayesian information criteria for the four models are given. From the g2 and AIC values it is 
seen that the hypothesized model fits the data better then the alternative models. Furthermore, 
Model A2 fits the data better than Model A3, which means that there may be a significant 
difference between the attraction of the alternatives for a subject in the "Don't know" state and a 
subject in the "Partial knowledge" state. The better fit of Model Aj relative to Model A j may 
indicate that a subject in the "Complete knowledge" state would make no mistake choosing the 
right alternative. 



Insert Table 2 here 



In Table 3 the estimates of the attraction parameters for the alternatives of each item are 
presented for the hypothesized model. These results indicate thai a subject in the "Partial 
knowledge" state is more likely to choose the correct alternatives to Items 1 and 4 than a subject 
in the "Don't know" state. Most likely, a test constructor would never expect that a subject in the 
"Don' I know" state is more likely 10 choose the correct alternative than a subject in the "Partial 
knowledge" state, as has happened for Item 2. The attraction parameter of the correct alternative 
of Item 2 in the first state is five times as large as the associated parameter in the second state. 
However, in both states the probability of choosing the right alternative is very low. In the 
"Partial Knowledge" state alternative A is almost always chosen, whereas in the "Don't know" 
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Slate altemaiive D is often chosen. Therefore* Item 2 has the advanuge of being possible to 
distinguish between subjects who exactly knew the solution of the problem imposed by Item 2 
and those who did not. The attraction parameters for the correct alternatives of Item 3 are 
approximately the same for both states. 

From Table 3 it can be seen that some of the attraction parameters are smaller than .05. 
An interesting question for these cases is if these attraction parameters are really unequal to zero 
or happen to yield estimates unequal to zero by chance. Although it is bad practice to formulate 
an alternative model post hoc after looking at parameter estimates and test it on the sample 
sample, we have tried to fmd an answer to the above mentioned question by fitting a fourth 
alternative model (A4), which had the same assumptions as the hypothesized model (H). It was 
also assumed that all the atuaction parameters with estimated value smaller tlian .05 in the 
hypothesized model were equal to zero. This alternative model showed a slightly improved fit 



Insert Table 3 here 



compared to the hypothesized model (see Table 2). It also followed that the alternative model fits 
the data better than all the other alternative models. In Table 4 the estimated attraction 
parameters for Model A4 are given. From this table it can be seen that the estimated attraction 
parameters in the case of Model A4 do not differ much from the estimated attraction parameters 
of the hypothesized model. 



Insert Table 4 here 
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Ai this point we have considered only the attraction parameters of the four items from the ASCP 
Medical Laboratory Test. In the remaining pan of this section we will taJcc a closer look at the 
difficulty parameters of the items. In this example it was assumed that the latent states were 
governed by the one-dimensional partial credit model. Using the scoring weights for this model 
as depicted in Figure 1(b) and omitting the latent trait index q, the one-dimensional version of 
Model (2) becomes 

exp ((e-5;(x))x) 

(10) P(xie) = 

Iexp((e -Sj(z))z) 



exp( I (B-Vjo)) 



z 

Iexp( I (e-Vjp)) 
z g=l 



where \|/jx = ^ Sj(x) - (x- 1 ) 5j(x- 1 ) describes the difficu j of step x in item j, because each latent 
state may be seen as the result of a series of subsequent steps, each of which has to be passed. In 
Table 5 the values of the vj^ parameters for the four items of the ASCP Medical Laboratory Test 
in the case of the hypothesized model H are given. 



Insert Table 5 here 



From Table 5 it follows that the difficulty of the steps changes positively for the items L 3 and 4. 
Meaning that it is more difficult to do the last step than the first step. On the other hand, for item 
2 it is difficult to take the first step, but if the first step is reached than the second step is very 
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easy. Following Verhelsi and Versiralen (1991)» two remarks have to be made. First, as 
Molenaar (1983) showed, the parameter value of a particular step depends on the parameter 
vaJues for the other steps in the item. Therefore, the parameter value of a step cannot be 
interpreted as a measure of its difficulty. Second, it cannot be known in advance that the items 
allow for a sequential solution as assumed in the partial credit model. 

Finally, some surprising results which are found during tlie analyses will be discussed. 
First from the 3370 subjects in the study no one had responded complete wrong to the four items. 
However, in the case of the hypothesized model it was estimated that 68 subjects had no 
knowledge of the solution of the problems imposed by all four items. On the other hand, almost 
327r of the subjects responded complete right to all four items, whereas it was estimated that 2% 
really knew all the solutions to the problems. 



Discussion 



In this paper a loglinear item response theory (IRT) model with latent classes was proposed that 
related polytomously scored item responses to a multidimensional latent space. The proposed 
model is a generalization of the solution-error response-error (SERE) model (Kelderraan 1988a; 
Westers & Kelderman, 1992) to situations of polytomously scored latent states tliat may be 
explained by a multidimensional latent space. In this generalized SERE model a distinction is 
made between some well-defined latent states in which the subject has a certain amount of 
knowledge of the answer. The probability that the subject is in a certain state is assumed to be 
governed by the multidimensional polytoraous latent trait model (MPLT). The relationship 
between the latent sutes and the observed answers is described by conditional probabilities. 

Maximum likelihood estimates of the parameters of the GSERE model can be obtained 
by the IFF algorithm. However, the results by Westers and Kelderman (1992) indicate that the 
(M)SERE model is usable in practice only when the responses to a few items are studied. 
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However, since the property of collapsibility is also valid for the GSERE model the DBI- 
principle of Westers (1992) can t>e used for estimating larger sets of parameters in the GSERE 
model. 

As pointed out in Westers and Kelderman (1992), an item can show DIP in two different 
ways. First, an item exhibits DIF if equally able individuals from different subgroups have 
different probabilities of knowing the answer. Second, an item also exhibits DIF if the 
attractiveness of the alternatives vary from subgroup to subgroup. Just as in the case of the SERE 
model, the GSERE model can be extended witli variables defining subgroups to study these two 
types of DIF. Therefore, the GSEEIE model is suitable for the examination of DIF in polytoraous 
items through a combination of DIF for correct/incomect responses and DIF in the alternatives. 
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Appendix A.l 

Two Items of the American Society of Clinical Patliologist (ASCP) Microbiology Test 

Item 11 Of the following bacteria, the most frequent cause of prosthetic heart valve 
infections occurring two to three months after surgery is: 

a. Streptococcus pneumoniae 

b. Streptococcus pyogenes 

c. Staphylococcus aureus 

d. Staphylococcus epidermidis 

Item 20 A beia-hemolytic gram-positiv coccus was isolated from the cerebroi.^ inal fluid 
of a 2-day-old infant with signs of meningtis. The isoltae grew on sheep blood 
agar under aerobic conditions and was resistant to a bacitracin disc. Which of the 
following should be performed for the presumptive identification of the 
organism? 

a. oxidase production 

b. catalase formation 

c. CAMP test 

d. esculin hydrolysis 
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Appendix A.2 
The Collapsed Generalized Solution-Error 
Response-Error Model 

In this appendix an instructional proof will be given that the GMSERE model in (7) is 
collapsible. The same notation will be used as in the previous sections. From elementary 
probability theory, it follows for the GSERE model that 

P(yi.y2) = 2: ... Z P(yi,...,yk) = 2: ... Z P(y) 

= ZZ...Z P(x)P(ylx)=: IP(x)I ... Z P(yix) 
xy3 yk X y3 yk 

= ZP(x)P(yilxi)P(y2lx2)Z ... Z P(y3lx3) ... PCyi^lx^) 

X y3 yk 

with P(x) is the marginal probability of the latent sute vector x. From conditional probability 
calculus it follows that 

P(yhy2)= Z P(x) P(yiixi)P(y2lx2) 

X 

= Z Z P(yilxi)P(y2lx2)Z ... Z P(x) 
xi X2 X3 xj^ 

Letting 2=(Z|,...,Zy) with Zq = BjqCxj) -f B2q(X2), and using the assumption of local 
independence of the latent states x; in the MPCM model and elementary calculus it follows that 
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I...Z P(x) = Z ...I J P(xie)dF(e) 
^3 "k ^3 



= fz...z n P(xjie)dF(e) 

"3 j=l 



=Jp(xiie)P(x2ie)dF(e) 



Using ihis last equation, the marginal probability of the observed responses yj and y2 can be 
written as 



P(yi,y2) = I I P(yilxi)P(y2lx2) f P(xil9) P(x2ie) dFO) 
XI X2 



= Z Z O^'T' «I>?2Y2 4,>^1 <i>X2 C)Tl2 

Ll X2 



X, xo "lyi "2X2 ''I "2 



with 



and 



*z'^ =Jexp(Z Zq9q){C(e.5i)C(e,82))-ldF(e) 



C(e.8j) = Zexp{ Z (eq-5jq)} 
z q=l 



^9 
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which is similar lo (7) except thai here we consider two items and in (7) k items. It may be clear 
that the collapsibiliiy of the GSERE model is also valid, if the general MPLT model is used for 
describing the dependence of the latent slates on ihe latent traits. 
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Table 1 

Hypothesized Weights of the ASCP M edical Laboratory Test 
Items for the Cognitive Process "Applying Knowledge". 



Item 




Scoring Weights 


Correct Answer 


A 


B 


C D 




1 






2 1 


C 


2 


1 




2 


c 


3 


2 


1 


1 


A 


4 


2 


1 


1 


A 



O ,3 

o4 
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Table 2 

) .ii-,-iihnnd-Ratio T"^' ^'i^tistics. AkaikT -' s Infor mation Critrrifl and Rafterv's Bavesiati 
Inf^r-^imr, Priirri p f"-- P""-- Ute ri s f''""^ 't^'- AS(-P Mrdiral Laboratory Test 



Model 


Number of 
Parameters 


g2 
Value 


AIC 
Value 


BIC 
Value 


H 

A4 


35 
53 
41 

29 
32 


197.557 
328.733 
190.259 
235.102 
200.283 


173.297 
291.996 
161.840 
215.001 
178.102 


-86.736 
-101.768 
-142.770 
-0.455 

-59.642 
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Table 3 

Attraction Parameters for the Alternatives of Four Items from the ASCP Medical Laboratory 
Test in the case of Model H 







Alternatives 






Alternatives 








"Don't know" state 




"Partial knowledge" 


state 


Item 


A 


B 


C 


D 


A 


B 


C 


D 


1 


.204 


.135 


.180 


.481 


.000* 


.000* 


ja6 


.324^ 


2 


.027 


.276 


■064 


.633 


.988 


.000 


■012 


.000* 


3 


.622 


.163 


.000 


.215 




.158 


.340 


.000* 


4 


.556 


.120 


.086 


.238 




.110 


.063 


.000 



Note 1. : The correct alternatives are underlined. 

Note 2. : Attraction parameters marked with an asterisk are prespecified. 
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Table 4 

Attraction Parameters for the Alternatives of Four Items from the ARC? Medical Lahnratnry 
Test in the case of ModRl Aj [ 







Alternatives 
"Don't know" state 




Alternatives 
"Partial knowledge" 


state 


Item 


A 


B 


C 


D 


A 


B 


c 


D 


1 
2 
3 
4 


.275, 
.000 
Ml 
■477 


.182 
.286 
.164 
.130 


■ 187 

.000 
.112 


■355 
.656 
.208 
.281 


.000* 
1.000 
.597 
■919 


.000* 
.000 
.124 
.081 


.000* 

.279 

.000 


.419, 
.000 
.000* 
.000 



Note 1.: The correct alternatives are underlined. 

Note 2f'- Attraction parameters maiiced with an asterisk are prespecified. 
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Figure Captions 

Figure 1 . Scoring Weights for the One- and Two-dimensional Partial Credit Model. 

Figure 2 . Examples of Specifications of the Attraction Parameters 

Figure 3 . Specifications of the Attraction Parameters of the Four Items from the 
ASCP Medical Laboratory Test for the Hypothesized Model H. 

Figure 4. Specifications of the Attracuon Parameters of the Four Items from the 

ASCP Medical Laboratory Test for the Alternative Models Aj, A2, and A3. 

Note 1 . y=A expresses the right alternative. 

Note L The specifications arc for all four items the same. 
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